enforcing lipschitz continuity
[D] Regularisation of Neural Networks by Enforcing Lipschitz Continuity
In this video we continue on the topic of Lipschitz continuity by presenting a paper which proposes a projection method to enforce it! If you enjoy this video consider watching others which I have on the topic! Abstract: We investigate the effect of explicitly enforcing the Lipschitz continuity of neural networks with respect to their inputs. To this end, we provide a simple technique for computing an upper bound to the Lipschitz constant---for multiple p-norms---of a feed forward neural network composed of commonly used layer types. Our technique is then used to formulate training a neural network with a bounded Lipschitz constant as a constrained optimisation problem that can be solved using projected stochastic gradient methods.